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Abstract 

For any n-tuple (ai, ■ • • ,a n ) of linearly independent vectors in Hilbert space 
H, we construct a unique orthonormal basis (ei, • • • , e n ) of span{a>i, ■ ■ ■ , a n } 
satisfying: 

£lh-«*ll 2 <E lift -^ll 2 

i=l i=l 

for all orthonormal basis • • • , f3 n ) of span{ai, • • • , a n }. We study the sta- 
bility of the orthornormalization and give some applications and examples. 
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1 Notations and Introduction 

Throughout this paper, 

1. C (or M.) is the complex (or real) number field. 

2. For any z G C, ~z is the complex conjugate of z. Re(z) is the real part of z. 

3. M nt m (C) = {(ay)} is the set ofnxm complex matrices. M n (C) = M n> n (C). 

4. C n = M n ,i(C). The identity of C" denoted by I n . 

5. For any (ay) G M n , m (C), (ay) = (ay) G M„ im (C). 

6. The standard basis of C n is denoted by = (5u, • • • , 5i n ),i = 1, • • • , n. 

7. For any (ay) G M n (C). ir(ay) = X^=i a ii- <T (( a ij)) i s the set of spectrum of 
(ay). ||(ay)|| = sup{ || (ay )j; t || : x G C n , ||x|| = 1}. 

8. if is a complex Hilbert space with the inner product (•, •). 

9. H n = {(«!,••• ,q„) : Oj G if, i = 1, ••• , n} is a Hilbert space, the inner 
product defined by: ((oq,--- , a n ), (/3i, • • • , /3 n )) = J^L^Oi, A)- 

10. For any (ay) G M n> m (C), (ay) T = (aji) G M m> n (C) and for any (ai, ■ ■ ■ , o„) G 



(ai, • • • ,a n ) = 
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If {ai, • • • , a n } C H n is subset of linearly independent vectors in H. The or- 
thonormalization of {a±, • • • , a n } is to find a solution {j3\, • • • , j3 n } in span{ai, • • • , a n } 
to the system: for any i,j, 

{ 1 i = j 

(PiiPj) = s ij = | o'i^j ' ^i = 1 '---> n - 

The orthonormalization can be carried out in infinitely ways. The most simple 
and practical tool is the Gram-Schmidt process. It is a recursive process and are 
used widely in various fields. However in general, the Gram-Schmidt process can not 
provide a simple construction formula for the solution and can not give a method 
to calculate the sum of squares 

n 

11(01,- •• ,/?„)- («!,••• ,a„)|| 2 = ^||ft-«,|| 2 - 

i=l 

Gram-Schmidt process is also unstable due to the repeated various operations. These 
restrict its applications, especially in the abstract or theoretical analysis. 

In numerical linear algebra, Householder method is also used in the orthornor- 
malization. The Gram-Schmidt process produces the jth orthogonalized vector after 
the jth iteration, while Householder method produces all vectors only at the end. 
And theoretically Household method take twice operations as Gram-Schmidt pro- 
cess, but it uses orthogonal transformation at each iteration, so it is stable. The 
Household method is restricted in numerical linear algebra only. 

The Gram-Schmidt process and Householder method can be find in Linear Al- 
gebra and Matrix Analysis text books, for example, see [1] or [2]. 

In Section 2, we provide a simple and uniform formula K(a\, • • • , a n ) (see (2.4)) 
for any Hilbert space, on complex field or real field, with finite or infinite dimensional, 
in numerical form or not in numerical form, no iteration, to construct an orthonormal 
basis of span{ai, ■ ■ ■ ,a n }, only according the direct information {(a^ay) : i,j = 
1, • • ■ , n}, satisfying: 

n n 

hi - "ill 2 = n + IHI 2 - 2tr i(( a i> «;)) 1/2 )- 
i=i i=i 

Moreover, we show n + Ya=i \\ a i\\ 2 ~ ^(((ccj, ay)) 1 / 2 ) is the lower bound of all sum 
squares of orthonormal basis of span{oti, ■ ■ ■ , a n }, and (ei, ■ ■ ■ , e n ) is the unique one 
minimized the sum of squares. 

In Section 3, we show our construction is stable in the sense: for any given e > 
and any linearly independent n-tuple (a±, • • • , a n ) in H n , there exists 5, dependent 
only on (a±, • • • , a n ) and e, such that for any • • • , j3 n ) £ H n , max{|[«j — < 6 
implies \\K{a.i, ■■■ ,«„)- K(P ± , ■ ■ ■ , (3 n )\\ < e. 

In section 4, as an application, we establish a formula for the distance of between 
any 7 G H and span{a\, • • • , a n }, generalize the one in the case {a±, • • • , a n } is an 
orthonormal basis. For any e-mutually orthonormal basis (a±,--- ,a n ),(see (4.4) 
for the definition), we show there is an orthonormal basis (ei, • • • , e n ), the distance 
between them in H n not exceed \/2(n — l)e. 



2 Least Square in Orthornormalization 

(1) For any («!,••■ , a n ) G H n , (ai, • • • , o n ) G C n , if 77 = £^ =1 a* a*, we write 



?? = (ai, • • • ,a n )(ai, • • • ,a n ) T = (ai, • • • ,a„) 



(2) If for any i, ft = (aji, • • • , aj n )(ai, • • • , a n ) T , i = 1, • • • , n, then we write 

/ «ii • • • ain \ ( oc\\ 



(Pi 



(2.1) 



That is 



ft = ei(aij)(ai,- ■■ ,a n ) ,i = 1, ••• ,ra. 
It is easy to check: if 77 = ^™ ajaj and £ = Sj=i &j/?j> then 



j=l i=l 



■31 



(&!,••■ ,6 m )((a j ,ft))(ai,--- ,a n ) T . (2.2) 



Theorem 2.1. Suppose (ai, • • • , a n ) is n-tuple of linearly independent vectors in a 
complex Hilbert space, then 

(1) For any orthonormal base (ft, • • • , (3 n ) of span{ai, ■ ■ ■ ,a n }, there exists an 
invertible (a^) G M n (C) such that (2.1) holds and satisfying, 

n n 

lift - «i|| 2 = n + INI 2 " 2Re(tr(((a j ,a i ))(a ji ))y (2.3) 
i=i i=i 

(£J ((ttj, aj)) is positive definite and if K(a±, • • • , a n ) = (ei, ■ ■ ■ , e n ) defined by 

(ai,ai) • • • (ai,«Ti) \ 1/2 /< 
(a„,ai) •• 



(a„,a„) / 



ai 



(2.4) 



then 



(a) (ei, ■ ■ ■ , e n ) is a orthonormal basis of span{ai, • • • , a ra }. 
(b) 



^ - ai\\ 2 = n + ^2 \\ a i\\ 2 - 2tr^({aj,ai)) 1/2 
i=i i=i 

(c) For any orthonormal base (ft, • • • , j3 n ) of span{a\, • • • , a n } 

n n 

^ll^-ailj 2 ^^Hft-^H 2 , 
i=i i=i 

and t/ie equality holds if and only if (ft, • • • , /3 n ) = (ei, • • • , e n ). 



(2.5) 



Proof. (1) There is no problem for the existence and the invertibility of (a^-). Notic- 
ing Pi = e i (a i j)(a 1 , ■ ■ ■ ,a n ) T and a* = ej(ai, • • • ,a n ) T , applying (2.2), we have 



EiiA-<*ii 2 

i=l 

= n + ^ ||aj|| 2 - ^2 + 
i=l i=l 

n n 

^||aj|| 2 -2i?e^^(ei(aij) (ai,--- , a„) T , e^ai, ■ ■ ■ ,a„) T }) 

i=l i=l 
n n 

^||a i || 2 -2i? e (^e l (((a„a i ))(a lj ) T )ef) 

i=l i=l 

= n + \\ a i\\ 2 ~ 2 Re(tr(((aj,ai))(aji))y 
i=i 

(2) (a) For any (a u ■■■ , a n ) € C n , not all zero, by (2.1) 

n ?i 

(ai,-- - ,a„)((a j ,a i ))(ai,--- ,a n ) T = ( ^ a^, ^ a;a:;\ > 0, 



n • 



= n 



i=l i=l 



so (ptj, ai)) is positive definite. Noticing ({ai, aj)) 1 / 2 = ({ai, ay)) 1//2 , and applying 
(2.1), we have 

(ei,ej) = e i ((a i ,a i ))" 1/2 ((a i ,ai))(((a i ,a i )) T )~ 1/2 ef = e,ef = 5 jU 

for any 

(b) (2.5) follows from (2.3) and (aji) = ((ay, a;))" 1 / 2 . 

(c) There exists a unitary J7 = (mj) € M n (C) such that 

un ■■■ Uln \ ( ei\ 

Unl ' ' ' U nn J / 



Then 



( 





-1 















( (on, Oil) 



\ (a„,ai) 



(«i,a„) \ 1/2 / ai 
(a n ,a n ) J 



a, 



VV 

so we have 

= (««)((«»> «j»~ 1/2 or = «ay>ai»" 1/2 («ji)- 

Therefore 

^e(tr(((a J ,a l ))(a, l ))) = Re(tr(({ aj , a*)) 1/2 («*))) ■ 

To the end of the proof, it is enough to show the statement: for any positive 
T G M n (C) and any unitary U, Re(tr(TU)^ < tr(T), and the equation holds if and 
only if U = I n . Since T may be assumed diagonal, it is easy to prove the statement. 
We omit the details. □ 



Remark 2.2. If H is a real complete inner product space, the conclusions in the 
Theorem 2.1 are still true. In fact, the inverse and squares root of a symmetric 
matrix are all real. Noticing (cti,aj) = {aj,ai), the proof has nothing to be changed 
except the unitary matrix replaced by a orthogonal matrix. 

Example 2.3. Suppose H = M. m , cti = (an, • • • , ai m ),i = 1, • • • , n. If (a±, • • • , a n ) 
is linearly independent, then n < m and 



(an ■ ■ ■ aim \ 



A 



\ a n i 



J 



has rank n, so AA T is invertible. If (AA T ) X I 2 A = (hij), then K(ai,--- ,a n ) 
(ei,--- ,e n ) defined by: = {ha,-- - ,h im ),i = !,■■■ ,n. 

For any other orthonormal basis (Pi, • • • , f3 n ) of span{ai, • • • , a n }, if 

fa = • • • ,b im ),i = 1, 

then 



,n, 



I>- 



Oti\ 



i=l 



Y Y ^ - hi 3 i 2 < Y Y \ hi i ~ hi 'j\ 2 = Y lift ~ ai 
j=i i=i j=i i=i 



i=i 



Example 2.4. Let H = L 2 ([0, 1]) be the space of real function which square is 
Lebesgue integrable. For any n, {l,x, • • • ,x n } is linearly independent. By the defi- 
nition of (2.4), 



fei\ 












w 





/ 1 



1 1 

2 3 
1 1 

I 3 

3 



_L_ \ 

n-l \ 



-1/2 



/ 1 \ 



X 
„2 



If the transformation matrix denoted by A n , then 

n 

Y ii a * ~ e *n 



i=i 



2tr(A- 1 ). 



For n = 4, with the help of MATLAB, we immediately get: 



ei = 


1.8145 




2.8273a; 


+ 2.0557a; 2 




0.6986a; 3 


C2 = 


-2.8273 


+ 


18.1940a; 


- 26.7823x 2 


+ 


11.9872a; 3 


£3 = 


2.0557 




26.7823a; 


+ 64.5308a; 2 




39.9282a; 3 


e 4 = 


-0.6986 


+ 


11.9872a; 


- 39.9282a; 2 


+ 


32.5816a; 3 



Ik* -1 - edl 2 = 4 + l + | + | + i- 2tr(A 4 x ) = 2.2201, 

i=i 

the accuracy is controlled by MATLAB. 



3 Perturbations 



In this section, we will discuss the perturbation problem and show our construction 
is stable in some sense. 

Note although ((aj,ay)) is non singular, it may be nearly singular when \\a>i\\ is 
too small. We may modify our work, replacing [a.\, • • • , a n ) by ( -^K , • • • , -rf\r) in 

1 1 Oil || || O77, 1 1 

this section. 

To show the main result of the section, we need the following Lemmas. Lemma 
3.1 is well known. 

Lemma 3.1. If A is an invertible element of unital C* -algebra A, for any B £ A 
with \\B — A\\ < ||A -1 || -1 , then B is invertible satisfying 

1 1 . , II A- II II ^4 II , 
I'- 4 ~ B U-\\\A-B\\ 

and 

n^^i-p-iP-^ir (3 ' 2) 

Lemma 3.2. If A, B are positive and invertible on a Hilbert space H , then 

p-1/2 _ £-i/2|| < ||^V2||p-i _ B -i\y (3.3) 

Proof. It is enough to show 

11-4 — -B|| < ||A _1 ||||^4 2 — i? 2 ||. (3.4) 

There exists A € a (A — B) with |A| = ||^4 — B\\ and {x n } C H, \\x n \\ = 1, i = 1, 2, • • ■ 
such that 

lim II (A — B)x n — Xx n \\ = 0. 

n— s-00 11 

Since A 2 - B 2 = A(A - B) + {A - B)A -{A- B){A - B), we have 

\\A 2 -B 2 \\ 

> liminfK^-S 2 ))^,^)! 

n— too ' 1 

= lhninf \(A(A - B)x n ,x n ) + ((A - B)Ax n ,x n ) - ({A - B)(A - B)x n , x n )\ 
= lim inf I (\Ax n , x n ) + (XAx n , x n ) - (X(A - B)x n , x n )\ 

n— >oo 

= liminf ||A||(^x n ,x n ) + (Bx n ,x n )\\ 

> \\A -BWWA-H- 1 , 



thus we obtain (3.4) and (3.3). □ 

The special conditions of the following Lemma 3.3 are prepared for the proof of 
Theorem 3.4. 

Lemma 3.3. Suppose A,B,C £ M n (C), B is positive, A is positive and invertible 
with \\A\\ < n, 1 < // there exists 77 with < rj < 2||^ _1 H -1 ), \\A - B\\ < rj 

and \\A — C\\ < rj, then B is invertible and 

WA-WCB- 1 ' 2 - I n \\ < 2n(n + l)\\A~ 1 \\ 2 r ! . (3.5) 



Proof. It follows Lemma 3.1, B is invertible and 

p- 1 - B- l \\ < 2||A- 1 || 2 ||A - B\\ < 2\\A- 1 \\ 2 r ] , (3.6) 

HE- 1 !! < 2||i4- 1 ||. (3.7) 
Condition p -1 !! > 1 implies [|^4_ — 1 1| > P~ 1/2 || and 

WB-^W = \\B- l \\ 1 ' 2 < (2||^1- 1 H) 1 / 2 < 2||^4- 1 ||. (3.8) 

By (3.3) and (3.6), 

_ B -i/2|| < pV2||p-i _ < 2||A 1 / 2 ||p- 1 || 2 ??. 

At last, applying (3.7) and (3.8), we get 

WA-WCB- 1 ' 2 - I n \\ 

< WA-WCB- 1 ' 2 - A^AB^W + \\A l ' 2 B- 1 ' 2 - A^A' 1 '^ 

< p-^IHIC - AWWB'^W + p 1 / 2 1| p- 1 / 2 - A- 1 ' 2 ]] 

< 2p- 1 || 2 7 ? + 2p||p- 1 || 2 r / 
= 2(n+l)p- 1 || 2 7 ? . 

□ 

Theorem 3.4. Suppose (a±, ■ ■ ■ ,a n ) is an n-tuple of linearly independent units of 
H n , for any given e > 0, let 

6= (snV + i)!!^-,^))- 1 !! 2 )^, 

then for any (Pi, • • • , j3 n ) of units in H n , if 

max{||Qj - /3j|| :i,j = l,--- ,n} < 5, (3.9) 

then • • • ,/3 n ) is linearly independent and satisfying 

\\K( ai ,--- ,a n )-K(pi,--- ,/3„)|| 2 <e. (3.10) 

Proof Let A = ({ctj,ai}),B = ({Pj, a;)), C = ({Pj,Pi}). Since tr(A) = n, so \ min < 
1, where \ m in = min [a (A)), this implies ||A _1 || = A~* n > 1. \\A\\ < n follows from 
for any \(ai,otj}\ < 1. The condition (3.9) guarantee for any 

| ( aj ,Oi)- {fa ,Pi)\\ < 25, | (aij , <*) - (aj , < 6- 

Consequently, 

\\A - B\\ < 2n5 < (2||^- 1 || 2 ) -1 , \\A - C\\ < 2nd. 
Let n = 2nd, then all conditions in Lemma 3.3 are all satisfied, so we have 

WA-^CB' 1 / 2 - I n \\ < 2(n + l)P _1 || 2 r? = 4n(n + \\ 2 S. 
The linearly independence of (Pi, - ■ ■ ,P n ) follows from the invertibility of B. 



Now suppose K(a 1 ,--- ,a n ) = (ei,-- - , e n ), K(/3i, ■ ■ ■ ,/3 n ) = (n,--- ,r n ), then 
by the definition (2.4) and formula (2.2), 

n n 

i=i i=i 
= 2n-f;2 J R e (e,((a J ,a l })- 1 /2 ((/ 3 J , a ,))(( /3j , ft ))-V2 e ^ 
i=i 

= 2Re(tr(I - A^^CB -1 / 2 )^ 

< 2\\I n - A^^CB-^Wtriln) < 8n 2 (n + l)||A- 1 || 2 ,5 = e, 
thus we complete the proof. □ 

4 Applications 

In this section, we will give some applications of our construction (2.4) to the theo- 
retical analysis. 

For a fixed linearly independent vectors {a±, • • • ,a n }, for any 7 G H, we define 
D(j) = dist(j, span{cti, ■■■ , a n }) = inf{||7 - : /3 G span{«i, • • • , a n }}. 
If {ai, • • • , a n } is mutual orthogonal units, then 



£>( 7 ) 



\ 



I 2 - 

i=l 



£i<7,«i)i 2 - 



The following theorem will show (4.1) is just (4.2) in the special case. 
For simple, in this section, for any (07, • • • , a n ) G H n , we define 

(«!,■■■ ,a„) o («!,-•• ,a n ) = ((a*, ay)) G M n (C). 

Theorem 4.1. Suppose (07,- ■■ , a n ) is n-tuple linearly independent vectors in a 
complex Hilbert space, then 



£)( 7 ) = / <fet((7,ai-- - ,a w ) 0(7, ,Qn)) ^ ^ 

y det((ai, • • • , a n ) o («!,-• • , a n )) 

Proof. Suppose (ei, ■■ ■ , e n ) = if(ai, ■ ■ ■ , a n ) defined by (2.4). Let A = det((ai, ay)), 
applying formula (2.2) and Y17=i eTe i = In, we have 



i=i 

n 

\T\2 



£l<7,< 

1=1 

n 

= £ l e *(("i' a *))" 1/2 (^' «i)> ■ ■ ■ , (7, a n )) 5 
i=i 

n 

= $^«ai>7)>-" ,(an,7))((ai,ai)) _1/2 efej((Q i ,Qj))~ 1/2 ((7,Qi),--- ,{^,a n )f 



i=l 



= ((ai,7),--- ,(a„,7))((aj,aj)) 1 ((7,ai)---(7,a„)) T . (*) 
Let Ay be the cofactors in ((ay,aj)) (not in ((««, «j))!), then 



(*) 



— «ai,7>,-" ,(an,7» 



Aln 



^ II SI 

^(5Z5Z^i<7,at)<aj,7>) 
j=i i=i 

/ ( 7 ,ai> • 
(ai,7) (ai,ai) • 



-1 



deb 



■ A nl 
A 



(7«i) \ 
(ai,a n ) 



'(7,"i) N 



\ (a n ,7) (a n ,ai) ••• (a n ,a„) / 
2 det((7,ai,- • • ,a n )o (7,01,- • • ,a n )) 



Since 



distil, span{ai, ■ ■ ■ ,a n }) 



^)l 2 , 



we obtain (4.2). 



□ 



Corollary 4.2. Suppose {a±, 02, • • • } is a sequence of independent vectors, then for 
any 7 G -ff, f/ie distance between 7 and i/ie closure of span{a±,a2, ■ ■ ■} is: 



lim 

n— >oo 



/ det((7,ai ■ ■ ■ ,a n ) o (7,01, • • • ,a n )) 
det((ai,- ■■ ,a n )o (a-i, ■ ■ ■ ,a n )) 



Lemma 4.3. Suppose E = {\i,i = 1, ■ ■ ■ , n} is a finite positive numbers set with 
Y^i=i = n an d max{|l — Aj| : i = 1, ■ ■ ■ , n} = e < 2 (n-i) > ^ en 



i=i 



1/2 



n 



< e. 



(4.3) 



Proof. We may assume Ai = min (E 1 ) and A ra = max (E 1 ) . 

If Ai = 1 or A n = 1, then for all i, Aj = 1, there is nothing to do. If A n = 1 + 77, 
then < 7] < e. Since the function 

n-2 n-2 

E^ + ^-^-E^) 172 

1=1 1=1 



on 



[1- 



-,1 + 



-]x..-x[l- 



1 + 



:1c 



on— 2 



2(n- ' 2(n- 1) J " " L ^ 2(n-l)' 2(n-l) J 
obtains its maximum value only when Aj = 1 — i = 1, ■ ■ ■ , n — 2, consequently, 

n n— 1 , /9 



i=i 



i=i 



If Ai = 1 — £, similar argument show 



Z>i /2 >n-c 



i=i 



Then (4.3) follows the assumption n < e and £ < e. 



If {ai, ■ ■ ■ , a n } satisfies condition, 



□ 



(4.4) 



e = max{|(aj,aj)| : i, j = 1, • • • ,n}, 

we will say {a%, ■ ■ ■ , a n } is e- mutually orthogonal. 

In [3], Hu and Xue proved, if {ati, • • • , a n } is e-mutually orthogonal, then there 
are mutually orthogonal {/3i, • • • , (3 n } with \\cti — All < 6(n — l)e, i = 1, • • • , n. Now, 
we have the following: 

Theorem 4.4. Suppose n-tuple units (a±,--- ,a n ) in if is e-mutually orthogonal 
with e < 2 {n-i) ' then t^re is an orthonormal basis (ei, • • ■ , e n ) of span{a\, ■ ■ ■ ,a n } 
such that 



ll e i ~ a if < 2 ( n - !) e - 



(4.5) 



Proof. Let T = ((ai, ctj)) and assume cr(T) = {Ai, • • • , A n }. Since (ctj, a^} = 1 and 

K a *' Q i)l < ( n - !k 

by Gersgorin Theorem ([2]Theorem 6.1.1), for all i, |Aj — 1| < (n — l)e. Meanwhile, 



i=i 



^ Aj = tr(T) = ^2(aci, a*) 

i=i 

then by the Lemma 4.1, 

tr(T 1 / 2 ) 



n. 



n 



8=1 



1/2 



n 



<(n- l)e. 



Let (ei, • • • ,e„) = if (ai, • • • ,a n ) defined by (2.4), applying (2.5) 

n n 

^|| ei -a i || 2 = 2|tr(T 1 / 2 )-n| = 2|^A 4 1/2 -n < 2(n - l)e 



i=i 



□ 
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